Optimal Pricing for an Unbounded Queue

The maximization of expected reward is considered for an M , / M / s queuing system with unlimited queue capacity. The system is controlled by dynamically changing the price charged for the facility’s service in order to discourage or encourage the arrival of customers. For the finite queue capacity problem, it has been shown that all optimal policies possess a certain monotonicity property, namely, that the optimal price to advertise is a non-decreasing function of the number of customers in the system. The main result presented here is that for the unlimited capacity problem, there exist optimal stationary policies at least one of which is monotone. Also, an algorithm is presented, with numerical results, which will produce an c-optimal policy for any e > 0, and an optimal policy if a simple condition is satisfied. Introduction The control of the operations of service facilities in order to maximize some economic gain function has been the subject of a considerable number of papers in the recent literature. The arrival process [ 1-81 and the service mechanism [7, 9151, are the two general areas of control which are usually considered, and since the latter is often more amenable to control, it has been the subject of most of the published work. It appears that the optimal control of a service facility in an open market has received somewhat less attention. By open market, we mean to describe a situation in which potential customers are free to take their business to any one of a number of competitors. One approach to modeling such a system is to allow an arriving customer to change his mind and leave if too many customers are already waiting for service [ I , 2 , 4, 6, 8, IO]. An alternate approach, more closely related to classical supplydemand relationships, is to allow the potential customer to choose whether or not to patronize the facility based on the current advertised price. It is not unreasonable to assume that the higher the price, the less likely it is that a given individual will buy the facility’s service. This assumption can be implemented by requiring the mean customer arrival rate (a reflection of demand) to be a decreasing function of price. The model is completed by the addition of a (possibly non-linear) holding cost which penalizes the facility for keeping its customers waiting. Motivation for this approach is supplied by Leeman [ 161, who discusses the concept of controlling queues through the use of price and cites several examples: “When an analyst in operations research encounters a queue, he seldom, if ever, looks into the alternative of introducing or changing a price in order to shorten or eliminate the queue. In practice, of course, prices often are used to reduce queues; examples are peak-load charges for electricity, higher daytime prices for parking, and higher Saturday prices for haircuts. But casual observation suggests that there are many unexplored, yet promising, possibilities of queue reduction through the use of price.” Two of the “possibilities” given in [ 161 are the use of price to “. . . reduce queues on congested highways and urban streets,” and the introduction of take-off and landing charges to reduce queues of aircraft waiting to take off or waiting (stacked) to land. One of the advantages Leeman cites for the price approach is an improvement in the allocation of existing service facilities: “Those who value services at particular points in space and time bid them away from others who value them less, so that scarce spatial-temporal bottlenecks are allocated to those who value them highly rather than on a first-come, first-served basis or on the basis of centrally established priorities.” One of the important applications of queuing theory today is in modeling and controlling the behavior of virtual memory computing systems. Here, an arriving customer represents a request for system resources (CPU time or space, 1 /0 facilities, etc.) and a service completion represents the partial or complete satisfaction of that request. A major problem area in the operation of a virtual system is the fact that user loads are often grossly LOW I IBM J. RES. DEVELOP. unbalanced over a day's time. When such a situation arises, the portion of time the system spends managing the queue (as opposed to doing useful work) increases drastically. This phenomenon is commonly known as "thrashing." A potentially valuable approach to this load balancing problem is to set prices for services which are based, at least in part, on the level of congestion, i.e., the length of the request queue. Such a policy would charge higher prices during peak load intervals, thus encouraging users to take advantage of the lower rates charged during low usage periods. Our objective is not simply the control of queues, but rather the maximization of reward through queue controls. The manager, in our model, must carefully balance the consequences of a price change. For example, if he increases the price at some point, the arrival rate of new customers is reduced but, on the positive side, the holding costs tend to decrease and, of course, each arriving customer pays more. Situations where the present model has some relevance are those in which the customer's primary motivation for selecting the given facility is price, not queue size (although price does give the informed customer a limited amount of queue size information). This preference for the price criterion over that of expected waiting time may come about in various ways: (a) the customer may be ignorant of the queue size; (b) he may be indifferent to the length of his expected wait; (c) he may have made significant personal or economic commitments before he discovers the length of the queue; or (d) he may be persuaded to remain in spite of the queue length. Examples might include cases where (a) a surrogate (an employee, a written purchase order, an application for a bank loan, a request for time or space in a virtual computer system, etc.) is sent to the facility instead of the actual customer; (b) the value of the customer's time is small compared to that of the service being purchased; (c) the customer hired a babysitter, paid a parking fee, walked several blocks and up three flights of stairs in order to get to the facility; and (d) the facility's holding cost is disbursed (in whole or in part) to the arriving customer as compensation for the inconvenience or expense' of his expected wait. The purpose of this paper is two-fold: first, to quantify the trade-offs mentioned above by means of a mathematical model, in order to obtain dynamic pricing policies which are optimal (or near optimal) ; and second, to ascertain the form of optimal policies. Underlying queuing optimization is the more general theory of Markov and Semi-Markov Decision Processes. Our work leans heavily on results of Fox [ 171, Lippman [ 181, and Ross [ 191 for the method used to prove the existence of optimal stationary policies. In the developJULY 1974 ment of our algorithm, the method of Ross [ 191 and Derman [ 201 is invaluable. The physical system can be described as an unbounded M , / M / s queue with variable arrival rate. The arrival process is Poisson with rate A, a strictly decreasing function of the currently advertised price p . The service times for the s servers are independent, exponentially distributed random variables with mean 1 / p . Control of the system is effected by increasing or decreasing the price p in order to discourage or encourage the arrival of customers. At each transition (customer arrival or service completion), the manager of the facility must choose one of a finite number of prices to advertise until the next transition. Definitions and system operation The queuing reward system described above can be modeled as a Semi-Markov Decision Process (SMDP) with action space P given by P = {pl, 1 . ., p K } , where 0 i p1 < p2 <. . . < p K < m, K < m, and state space 1 = (0 , 1, . . .}. Here, K is the number of prices available to the manager of the facility. The number of servers in the system is denoted by s. If the manager choses action p E P when the system is in state i, the transition probabilities are given by 4 i , i + l ( ~ ) = A,/ [ (i A + A,], (1 ) 4 i + l , i ( P ) = 1 4i+l,i+z ( P I , i = 0, 1, . . ., ( 2 ) where 0 < A <. . ' < A < sp, and where (i A s) denotes the minimum of i and s. The net reward received immediately following a customer arrival when there are i customers in the system is p ci; p is the currently advertised price and ci represents a holding cost. The assumption that A,, < sp ensures that all states are positive recurrent. The assumption that the holding cost ci is a lump sum (associated with an arriving customer) is made for computational convenience. It is innocuous since the long range average rate of return per unit time will not be influenced by when (during the customer's stay in the system) this cost is assessed. The holding cost function c : 1 + 9 is assumed to satisfy P& PI 0 5 c O = c 1 = " ' = c,-l 5 c, i ' . ' < pr (3 1 The equalities above are reasonable since all customers who arrive when there is at least one server free have the same expected time in the system, namely, 1 / p . We require the costs to be bounded by p K in order to ensure the possibility of a positive reward for each arriving customer. There is no cost or reward associated with a departure (service completion). If, when in state i, action i